AMENDMENTS TO THE DRAWINGS : 

The attached Replacement Sheets include changes to Figures 1 and 2. These sheets 
replace the original sheets containing Figures 1 and 2. No new matter has been added. 
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REMARKS 



This Preliminary Amendment cancels, without prejudice, claims 1-10 in the 
underlying PCT Application No. PCT/EP2004/052027 and adds new claims 1 1-21. The new 
claims, inter alia , conform the claims to United States Patent and Trademark Office rules and 
do not add any new matter to the application. 

In accordance with 37 C.F.R. § 1.125(b), the Substitute Specification (including the 
Abstract) contains no new matter. The amendments reflected in the Substitute Specification 
(including Abstract) are to conform the Specification and Abstract to United States Patent 
and Trademark Office rules or to correct informalities. As required by 37 C.F.R. §§ 
1.121(b)(3)(ii) and 1.125(c), a Marked-Up Version of the Substitute Specification comparing 
the Specification of record and the Substitute Specification also accompanies this Preliminary 
Amendment. Approval and entry of the Substitute Specification (including Abstract) are 
respectfully requested. 

The underlying PCT Application No. PCT/EP2004/052027 includes an International 
Search Report, dated December 22, 2004, a copy of which is included. The Search Report 
includes a list of documents that were considered by the Examiner in the underlying PCT 
application. 

It is respectfully submitted that the subject matter of the present application is new, 
non-obvious, and useful. Prompt consideration and allowance of the application are 
respectfully requested. 



Respectfully Submitted, 





G^tr^A. Messina 
Reg. No. 35,952 



One Broadway 

New York, NY 10004 



(212) 425-7200 
(212) 425-5288 



CUSTOMER NO. 26646 



1076946 
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[10191/4260] 

METHOD FOR EVALUATION AND STABILIZATION OVER TIME OF 

CLASSIFICATION RESULTS 

Background Of BACKGROUND INFORMATION 

The Invent ionThc present invention prococdo f ro m relates to a method 
and system for evaluating classification results from 
classification methods which proceed in computer-assisted fashion 
5 and in which objects to be classified are sensed using sensors over 
a period of time, and are repeatedly classified with the inclusion 
of specific quality parameters for an object class. A quality 
parameter of this kind is a parameter number or another qualifying 
indication within a preselected range, and is expressive of the 
10 certainty that the object belongs to the particular object class 
and not to a different object class. 

Observations of objects are evaluated today in many application 
sectors, e.g.^ in machine vision in industrial production 

15 processes, in safety-related applications, for sensing the 

surroundings of vehicles, etc. These observations are furnished 
by measurement devices that can comprise, in principlc for example , 
one or more sensors that in some cases is different, e.g._^ radar, 
ultrasound, video in the visible and infrared region, lidar, laser, 

2 0 range imager . 

One goal when processing object data obtained in this fashion is 
classification. "Classification" is understood to be the 
allocation of objects to specific object classes. A class serves 

25 to describe multiple objects that are considered, on the basis of 
similar features, to belong together. Classification systems for 
the assignment of objects to object classes are used, for example, 
in industrial production. Sensing and object classification of the 
surroundings of a motor vehicle while driving also promises great 

30 benefits . 
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Classification according to known conventional classification 
methods has the disadvantage that, especially in borderline cases 
with regard to allocation of the sensed object to a class, an 
5 elevated risk of misclassif ication exists. In borderline cases, 
for example, there is a high probability that in the context of 
individual classification results, the object will be attributed 
to the wrong object class. Outliers in the context of 
classification of an object, resulting^ e.g.^_ from measurement 
10 errors, also have a negative effect on the stability and robustness 
of prcocntly known conventional classification methods. 



The conventional methods according to the cxioting art furthermore 
offer only a yes/no classification of the sensed object. These 

15 results are insufficient for situations that require a more 

differentiated classification result, for example in the context 
of estimating accident hazards for a motor vehicle. One reason for 
this is that in the context of a classification repeated over time, 
an object is allocated first to one class and then, as the object 

20 is tracked, later on to a different class; this instability means 
that a clearly interpretable result is not supplied. Information 
as to how certain it is that the object might not also belong to 
one of the other classes would therefore be additionally important. 

25 Advantage a Of The Invention 
SUMMARY 

The foaturco of the independent claimo rcoult in ln accordance with 
example embodiments of the present invention, a stabilization over 

30 time of the classification may result. According to the present 
invention, a confidence parameter is calculated from the quality 
parameters of the individual measurements, and is generally 
"integrated" over a certain number of time steps k and 
corresponding individual measurements. An abruptly large change 

35 in the confidence parameters for each object class with reference 
to successive time steps is thereby avoided, so that individual 
outliers during classification can be filtered out. A 
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strengthening or weakening of the classification result is also 
effected when a subsequent classification of the result confirms 
or does not confirm a previous one. "Confirmation 11 is understood 
here as at least a tendency toward allocation of the object to the 
5 same class. For example, a classification with reference to an 
object class as the third most suitable class among ten classes 
can also represent a confirmation for a previous classification 
of the object in that class. 

10 Because, according to fefe ean example embodiment of the present 
invention, the method continuously repeats, the classification 
result in borderline cases is stabilized to the somewhat more 
probable object class, so that disruptive back-and- forth jumps in 
the classification result are suppressed and the method is robust 

15 in terms of such disruptions. 

The utilization of time-related information in the context of the 
present invention furthermore makes possible a more differentiated 
evaluation of the classification result. This kind of "final 
20 classification result, " based on time-related context information 
and as a function of confidence parameters adapted by way of many 
measurements, can yield a substantially better result than a 
classification based on individual results. 

2 5 The dependent claims dcocribc advantagcouo embodiments and 

rcfincmcnto of, and improvemcnto to, the respective oubject matter 
of the invention. 

One advantageous refinement of the example method according to the 

3 0 present invention is a dependence of the increase and/or decrease 

in the value of the respective confidence parameter on its absolute 
quality, the "absolute quality" of the confidence parameter being 
understood as its level. With a high confidence parameter, for 
example, a value increase can be correspondingly small so as not 
35 to create too great a difference between the confidence parameters, 
since too great a difference would otherwise interfere with an 
inherently correct correction of a misclassif ication . 
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This problem is also solved in another refinement of the method 
in which the permissible value range for the confidence parameters 
is limited. 

5 An additional improvement to the example method according to the 
present invention is represented by weighted inclusion of the 
absolute quality of the respective individual classification 
results, with reference to the individual object classes, into the 
increase and/or decrease of the corresponding confidence parameter. 
10 The reliability of the individual results for each object class 
is thus also included in the classification result. A higher 
classification quality is the consequence. 

A further advantageous method step is that of evaluative analysis 
15 of the calculated confidence parameters to determine a final, 

detailed classification result. It is thereby possible to arrive 
at more-differentiated statements regarding classification, for 
example the statement that the object fits badly into class A and 
fits very badly into all other classes. This is an improvement over 
20 the statement that "class A is the best fit." 

Also advantageous are on the one hand an evaluation constituting 
a classification into a higher-order class in the event the 
classification results alternate between similar object classes 

25 (example: if the result alternates between "small car" and "large 
car, " the final result might be the higher-order class "car") , and 
on the other hand an evaluation constituting a refusal to classify 
if the classification results jump back and forth over time between 
dissimilar object classes (example: if the result alternates 

3 0 between "truck" and "pedestrian, » the final result might be "object 
cannot be reliably classified"). 

The method according to the present invention can be used, in 
particular, to evaluate the classification results of a 
3 5 classification method for objects in the surroundings of a vehicle. 
This can be accomplished using a computer-assisted vehicle 
information system that contains the following devices: 
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- effective connections to relevant interfaces to vehicle sensor 
devices for sensing objects in the vehicle's surroundings ; 

- a control circuit for analysis and classification of the sensed 
5 objects, suitable for carrying out the method according to the 

present invention; and 

- effective connections to relevant interfaces to actuator devices 
on the vehicle. 

10 

Drawingo 

BRIEF DESCRIPTION OF THE DRAWINGS 

Excmplifying Example embodiments of the present invention will 
15 beare explained below with reference to the drawings, — ki 
which ; figures . 

Figure 1 is a block depiction of a computer-assisted vehicle 
information system set up to carry out an example embodiment of 
2 0 the method according to the present invention^ 

Figure 2 is a flow chart for an example embodiment of the method 
according to the present invention-; — a**€L_ 

25 Figure 3 is a sketch of a scene in the surroundings of a vehicle, 
with results from an example embodiment of the method according 
to the present invention. 

Dcocription Of Exemplary Embodiments 

30 

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS 

In the Figures, identical reference characters denote identical 
or identically functioning components. 

35 Referring to Figure 1, sensors 5 for sensing objects (e.g. trees, 
pedestrians, passenger cars, or trucks) in the surroundings of 
vehicle 1 are mounted in and/or on vehicle 1, for example a 
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passenger car. The nature of sensors 5 can be, for example, radar, 
ultrasound, video in the visible and infrared region, lidar, laser, 
and range imager camera. 

5 The measured data acquired by the sensors regarding a sensed object 
are forwarded via effective connections and interfaces (e.g. cable, 
radio, infrared transmission) to vehicle information system 3 
integrated into vehicle 1. Vehicle information system 3 
pooocooco includes a computer unit 7 associated with which is a 
10 database 9 that contains data about features regarding object 
classes such as pedestrians, transport vehicles, and passenger 
cars . 

Also part of computer unit 7 is a control circuit 11 for classifying 
15 the sensed objects and evaluating the classification result. Both 
the measured data from sensors 5 and the data about the features 
of the object classes required from database 9 are fed into this 
control circuit 11, and processed therein for classification. 
According to an example embodiment of the present invention, the 
20 confidence parameters, which indicate how well the sensed object 
matches the particular object class, are correspondingly increased 
or decreased in value depending on whether the previous 
classification is or is not confirmed, e.g. ± raised by three points 
per iteration, or lowered by three points for a decrease in value. 

25 

According to fetean example embodiment of the present invention, 
a programmed evaluative analysis of the classification result also 
occurs in computer unit 7. This results in a differentiated 
classification, for example the statement that the object fits very 
30 well into class A and fits moderately well into all other classes. 

When the classification result has achieved sufficient stability 
over time according to an example embodiment of the present 
invention, specific actuator devices 13 of vehicle 1 can be 
35 triggered by control unit 7, if desired, via effective connections 
(e.g. cable, radio, infrared transmission); for instance, the 
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brake can be actuated if the classification result is used, for 
example, in conjunction with a collision prediction and a safety 
system. 

5 Figure 2 depicts an example embodiment of the method according to 
the present invention as a flow chart. 

A preclassif ication 100, which is based on a single measurement 
using the aforementioned sensors and a single interpretation of 

10 the measurement results obtained therefrom, and does not yet 

contain any stabilization over time and does not yet represent a 
confidence parameter ao defined in thio Application described 
herein , supplies, for each object to be investigated, parameter 
numbers constituting a "quality parameter" ao defined above, for 

15 matching to all the classes that are to be distinguished. Motivated 
by information theory, what is used in this example as the parameter 
is a so-called word length, which should be short in the event of 
a good agreement between object and class. This "classification 
word length" is referred to hereinafter as "w_i," where i is the 

20 index for a class. 

As a possible enhancement, a "rejection word length" w_rej_i can 
be calculated in addition to the classification word lengths w_i . 
This parameter refers to the question of whether the object does 

25 NOT fit into the particular object class and is to be rejected. 
The calculation of w__rej_i can proceed, in principle, differently 
from that of w_i . Both preclassif ication results (w_i and w_rej_i) , 
or conversely only one of the two, are used in this example as input 
variables of the example method according to the present invention; 

30 i.e._^ the aforementioned "quality parameter" serves for 

calculation of the actual confidence parameter, as described in 
more detail below. 

For the object to be classified, a so-called hypothesis exists for 
35 each of the object classes to be distinguished. Hypothesis i is 
"the object belongs to class i." 
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The probability that a hypothesis is true is evaluated using the 
confidence parameter "conf_i. " The method according to the present 
invention uses these confidence parameters to evaluate all the 
5 classification word lengths. 

In addition to the input data mentioned above, according to the 
present invention the evaluation system uses the confidence 
parameters from previous time steps (k-1) , i.e. "conf_i_k-l , " for 
10 each of the individual hypotheses. 

In processing step 110 (adaptation of the confidence parameters) , 
a distinction 120 as to specific cases is made. These cases can 
be handled differently depending on the application, i.e.^ the 
15 system can in principle also be used without a distinction of cases, 
or with fewer or more distinctions. In this example the following 
three cases are distinguished: 

a) The object was classified in the current time step, and not 
2 0 rejected. 

b) The object was categorized by the classifier in the current 
time step as belonging to none of the object classes, and 
therefore rejected. This can be done, for example, by 

25 comparing the rejection word lengths w_rej_i for all the 

classes with a defined rejection threshold. 

c) No classification word lengths (w_i and/or w_rej_i) exist for 
the object in the current time step. This can be attributable 

30 to insufficient classifier input data, but can also result 

from deliberate lack of processing of the object. It may be 
useful, for example, in the interest of high processing speed, 
not to reclassify in the current time step an object that 
already has a high confidence parameter, but instead to 

35 perform the classification only on the basis of the old 

confidence parameters . 
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Case a) - The object was classified in the current time step : 

According to the classification word lengths w__i, an increase in 
5 the value of the corresponding confidence parameter muot bc is 
performed. One possibility is to increase the value according to 
a quality sort of the individual hypotheses: all the word lengths 
w_i are sorted in ascending order, and the smallest w_i indicates 
class that "best matches" the object. The confidence parameters 
10 for each hypothesis are increased in value in accordance with this 
ranked list. The formula might be as follows: 

confidence parameter for the class in first place on the list 
is increased in value by 5 points; 

15 

confidence parameter for the class in second place on the list 
is increased in value by 3 points; 

confidence parameter for the class in third place on the list 
2 0 is increased in value by 1 point; 

all other confidence parameters remain unchanged. 

The number of classes to be taken into account, and the relationship 
25 among the value increases, can be selected without restriction 
based on the particular application. 

The increase in value is additionally performed as a function of 
the absolute quality of the individual result. Here an additional 

30 weighting factor g___i is calculated for each class (see step 130) . 
This factor takes into account the quality of the result for each 
class (i.e. the level of w_i in relation to a specific threshold) . 
The distance from w_rej_i to the rejection threshold may be used 
here as an example : this distance is an indication of the evaluation 

35 "the object does not belong to any of the classes." 
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g_i = max (0; 1 - (w_rej_i / rejection threshold)) 

For objects that were classified but almost rejected, the weighting 
factor assumes a value close to 0 . This means that the corresponding 
5 hypothesis was in fact barely confirmed, and an increase in the 
value of the confidence parameter is therefore not plausible. For 
a higher-quality result (corresponding to a greater distance from 
the rejection threshold), g_i approaches 1. This applies to the 
assumption of a positive w_rej_i. 

10 

The result for the increase in value of the confidence parameters 
is then: 5 * g_i for the class in first place, and 3 * g_i and 1 
* g_i correspondingly for second and third place, respectively. 

15 Case b) - The object was "rejected" by the classifier in the current 
time step : 

In the event of rejection, the confidence parameters of all the 
hypotheses are decreased in value: a rejection of the object means 
20 that the object cannot be allocated to any of the defined classes, 
and therefore none of the defined hypotheses can be supported. 

Correlated with the quantitative example of a 5 , 3, 1 value increase 
cited in case a) , a value decrease of, for example, 4 for all 
25 confidence parameters can be applied in case b) . 

As in case a) , a quality-dependent additional weighting factor g2_i 
is used for the value decrease in the confidence parameters for 
each class i (130) . This factor can once again, for example, be 
30 based on the distance between the rejection word length w_rej_i 
and a defined rejection threshold: 

g2_i = min (1; [1 - (w_rej_i / rejection threshold)]) 

35 In this case g2_i assumes large values (a maximum of 1) for a large 
distance from the rejection threshold: the more unequivocally the 
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object was rejected, the more the value of the confidence parameter 
must be decreased. For a w_rej_i close to the rejection threshold, 
weighting factor g2_i approaches 0 (no value decrease) . 

5 Case c) - No classification result exists for the object in the 
current time step : 

If the classification word lengths (w_i and/or w_rej__i) are absent, 
all the confidence parameters are decreased in value by a specific 

10 identical amount. This represents a reduction in the certainty that 
the hypotheses are correct. For the quantitative example being used, 
a decrease in the value of all confidence parameters by 1 is 
selected. It is obvious in this context that the absolute numbers 
should be coordinated with the classification iteration rate, the 

15 value range limits, and the application of the method. The 

iteration rate here is the frequency with which individual quality 
parameters are calculated, per predefined unit of time, in the 
aforesaid time steps k. 

20 Here as well, it is advantageous to apply a more differentiated 
value decrease as a function of various cases (e.g.^ by asking "why 
was the object not classified," etc.). 

Adapting the value range (see step 14 0) : 

25 

An adaptation 140 of the value range of the confidence parameters 
is effected by limiting them. Here as well, it io obvioua that the 
absolute numbers should be coordinated with the classification 
iteration rate and the application of the method. 

30 

The value range does not necessarily have to be symmetrical about 
zero; a smaller range for negative confidence parameters and a 
larger range for positive ones may prove useful depending on the 
application. In this example a range from -20 to 200 is selected. 

35 
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If the confidence parameters depart from this value range as a 
consequence of an increase or decrease in value, an adaptation is 
necessary; otherwise a classification would no longer be possible, 
since all (or at least several) of the hypotheses would be supported 
5 to the same extent. This can be done differently for a value range 
overflow and underflow. One possible adaptation is: 

A value range underflow causes the confidence values to be 
truncated. If a confidence parameter is on the verge of 
10 dropping below this level, it is set to the minimum possible 

value . 

A value range overflow causes the confidence parameters not 
to be simply truncated, but to be adapted in accordance with 

15 their relationship to one another. When a confidence 

parameter reaches the maximum value, for example, a value 
decrease can be performed for all the other confidence 
parameters by the same amount by which the high confidence 
parameter would normally be increased in value. The distance 

2 0 between the high confidence parameters and the other 

confidence parameters is thereby maintained. 

The aforesaid adaptation is only an example of the plurality of 
possible procedures . 

25 

Determining the final classification result (step 150) : 

According to the present invention, a determination 150 of the 
classification result for the object in question is performed as 
30 the last processing step of the method according to the 

cxcmplif ying example embodiment. This result is ascertained on the 
basis of the calculated confidence parameters. 

A very large number of possibilities exist here for detailed 
35 evaluation, for example: 



NY01 1076872 vl 



12 MARKED UP COPY OF THE 

SUBSTITUTE SPECIFICATION 



Assign the object to the class for which the highest 
confidence parameter exists. 

As an additional alternative: the classification result is 
5 valid only if this highest confidence parameter exceeds a 

given minimum threshold. 

As an additional alternative: the classification result is 
valid only if the distance between this highest confidence 
10 parameter and the second-highest confidence parameter 

exceeds a defined threshold (otherwise the result is 
characterized as "not unequivocal"). 

For stabilization over time according to the present invention, 
15 the confidence parameters are temporarily stored and are used as 
input data for the next processing step of the method. 

Figure 3 depicts a scene in the surroundings of a vehicle, showing 
the results of an example embodiment of the method according to 

20 the present invention. For each of the objects present in the scene 
(vehicles only, in this case) , one of which is labeled with 
reference character 206, icons 200 show the class of the object 
by way of a symbol 2 02, and indicate the level of the confidence 
parameters using a bar 204. This depiction is not intended for the 

2 5 driver, but serves only for further explanation of the present 
invention . 

Although the present invention has been described above on the 
basis of a preferred cxcmpl i f ying exampl e embodiment, it is not 
30 limited thereto but rather is modifiable in numerous ways, e.g._j_ 
the absolute magnitude of the value increase/decrease delta 
numbers, the iteration duration, or the limits. 

The method according to the present invention can be used to improve 
35 any type of classification. The method encompasses a plurality of 
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steps and variants that for the most part are mutually independent, 
and it can be flexibly configured. 

Lastly, the features of the dependent claimo present invention can 
be combined with one another substantially without restriction and 
not in the ocqucncc cxioting in the Claimo, — provided they arc 
independent of one another . 
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Aba tract A BSTRACT 



The invention conccrna claooif ication Classif ication methods 
(100) are described that proceed in computer-assisted fashion, and 
5 in particular a method for evaluation and stabilization over time 
of classification results is described in which objects to be 
classified are sensed using sensors over a period of time, and are 
repeatedly classified with the inclusion of specific quality 
parameters for each object class. To ensure better classification 
10 reliability, the following steps aremay be carried out: 

a) increasing the value (110 , — 120) of the confidence parameter 
if a subsequent classification confirms the result of a previous 
classification; 

b) decreasing the value (110 , — t2£-) — of the confidence parameter 
15 if a subsequent classification does not confirm the result of a 

previous classification; 

and c) generating (150) a final classification result including 
the confidence parameters that have been increased or decreased 
in value. 
2 0 (Figure 2) 
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